Exploiting Symmetries in Two Player Zero-Sum Markov Games with an Application to Robot Soccer

The exploitation of symmetries in Markov decision processes has proven to be a powerful concept for state space reduction without compromising solution optimality. In this paper we show how to perform a symmetry reduction for two player zero-sum Markov games with additional symmetry properties. The concept of equivariance symmetry justifies for the first time not only to lift the standard MDP symmetries to the Markov game case but also a qualitatively new symmetry: the exchange of two adversary players. The proof technique also deviates from the standard approach, for it directly utilizes the uniqueness of Bellman’s equation. We apply our reduction procedure to a multi player robot soccer model.